Riesgos de interpretación errónea en la evaluación de la Supervisión Distante para la Extracción de Relaciones

Distant Supervision is frequently used for addressing Relation Extraction. The evaluation of Distant Supervision in Relation Extraction has been attempted through Precision-Recall curves and/or calculation of Precision at N elements. However, such evaluation is challenging because the labeling of the instances results from an automatic process that can introduce noise into the labels. Consequently, the labels are not necessarily correct, affecting the learning process and the interpretation of the evaluation results. Therefore, this research aims to show that the performance of the methods measured with the mentioned evaluation strategies varies significantly if the correct labels are used during the evaluation. Besides, based on the preceding, the current interpretation of the results of these measures is questioned. To this end, we manually labeled a subset of a well-known data set and evaluated the performance of 6 traditional Distant Supervision approaches. We demonstrate quantitative differences in the evaluation scores when considering manually versus automatically labeled subsets. Consequently, the ranking of performance among distant supervision methods is different with both labeled.La Supervisión Distante se utiliza con frecuencia para abordar la extracción de relaciones. La evaluación de la Supervisión Distante en la Extracción de Relaciones se ha realizado mediante curvas de Precisión-Cobertura y/o el cálculo de la Precisión en N elementos. Sin embargo, dicha evaluación es un desafío porque el etiquetado de las instancias es el resultado de un proceso automático. En consecuencia, las etiquetas no son necesariamente correctas, afectando no solo el proceso de aprendizaje sino también la interpretación de los resultados de la evaluación. El objetivo de esta investigación es mostrar que el desempeño de los métodos medido con las estrategias de evaluación mencionadas varía de manera significativa si se utilizan las etiquetas correctas durante la evaluación. Además, basado en lo anterior, se cuestiona la interpretación actual de los resultados de estas medidas. Con este fin, etiquetamos manualmente un subconjunto de un conjunto de datos y evaluamos el desempeño de 6 enfoques tradicionales de Supervisión Distante. Demostramos diferencias cuantitativas en los puntajes de evaluación al considerar subconjuntos etiquetados manualmente versus automáticamente. En consecuencia, el orden de desempeño entre los métodos de Supervisión Distante es diferente con ambos etiquetados.The present work was supported by CONACyT/México (scholarship 937210 and grant CB-2015-01-257383). Additionally, the authors thank CONACYT for the computer resources provided through the INAOE Supercomputing Laboratory’s Deep Learning Platform for Language Technologies

Monitoring attentional processes for intelligent channelling of educational tasks

Aims: - Detection of attention: Map a lexicon of body postures to binarizedattentionallevels (Experiment I). - Attribution of attention: Identify postural features leading to appreciation of attention by third parties (e.g. educators) (Experiment II).Panel ACE.Ibero-American Science and Technology Education Consortium (ISTEC

Effect of the level of task abstraction on the transfer of knowledge from virtual environments in cognitive and motor tasks

Introduction: Virtual environments are increasingly being used for training. It is not fully understood what elements of virtual environments have the most impact and how the virtual training is integrated by the brain on the sought-after skill transference to the real environment. In virtual training, we analyzed how the task level of abstraction modulates the brain activity and the subsequent ability to execute it in the real environment and how this learning generalizes to other tasks. The training of a task under a low level of abstraction should lead to a higher transfer of skills in similar tasks, but the generalization of learning would be compromised, whereas a higher level of abstraction facilitates generalization of learning to different tasks but compromising specific effectiveness.// Methods: A total of 25 participants were trained and subsequently evaluated on a cognitive and a motor task following four training regimes, considering real vs. virtual training and low vs. high task abstraction. Performance scores, cognitive load, and electroencephalography signals were recorded. Transfer of knowledge was assessed by comparing performance scores in the virtual vs. real environment.// Results: The performance to transfer the trained skills showed higher scores in the same task under low abstraction, but the ability to generalize the trained skills was manifested by higher scores under high level of abstraction in agreement with our hypothesis. Spatiotemporal analysis of the electroencephalography revealed higher initial demands of brain resources which decreased as skills were acquired.// Discussion: Our results suggest that task abstraction during virtual training influences how skills are assimilated at the brain level and modulates its manifestation at the behavioral level. We expect this research to provide supporting evidence to improve the design of virtual training tasks./

Optical neuroimaging and neurostimulation in surgical training and assessment: A state-of-the-art review

IntroductionFunctional near-infrared spectroscopy (fNIRS) is a non-invasive optical neuroimaging technique used to assess surgeons' brain function. The aim of this narrative review is to outline the effect of expertise, stress, surgical technology, and neurostimulation on surgeons' neural activation patterns, and highlight key progress areas required in surgical neuroergonomics to modulate training and performance.MethodsA literature search of PubMed and Embase was conducted to identify neuroimaging studies using fNIRS and neurostimulation in surgeons performing simulated tasks.ResultsNovice surgeons exhibit greater haemodynamic responses across the pre-frontal cortex than experts during simple surgical tasks, whilst expert surgical performance is characterized by relative prefrontal attenuation and upregulation of activation foci across other regions such as the supplementary motor area. The association between PFC activation and mental workload follows an inverted-U shaped curve, activation increasing then attenuating past a critical inflection point at which demands outstrip cognitive capacity Neuroimages are sensitive to the impact of laparoscopic and robotic tools on cognitive workload, helping inform the development of training programs which target neural learning curves. FNIRS differs in comparison to current tools to assess proficiency by depicting a cognitive state during surgery, enabling the development of cognitive benchmarks of expertise. Finally, neurostimulation using transcranial direct-current-stimulation may accelerate skill acquisition and enhance technical performance.ConclusionFNIRS can inform the development of surgical training programs which modulate stress responses, cognitive learning curves, and motor skill performance. Improved data processing with machine learning offers the possibility of live feedback regarding surgeons' cognitive states during operative procedures

Collaborative Gaze Channelling for Improved Cooperation During Robotic Assisted Surgery

The use of multiple robots for performing complex tasks is becoming a common practice for many robot applications. When different operators are involved, effective cooperation with anticipated manoeuvres is important for seamless, synergistic control of all the end-effectors. In this paper, the concept of Collaborative Gaze Channelling (CGC) is presented for improved control of surgical robots for a shared task. Through eye tracking, the fixations of each operator are monitored and presented in a shared surgical workspace. CGC permits remote or physically separated collaborators to share their intention by visualising the eye gaze of their counterparts, and thus recovers, to a certain extent, the information of mutual intent that we rely upon in a vis-à-vis working setting. In this study, the efficiency of surgical manipulation with and without CGC for controlling a pair of bimanual surgical robots is evaluated by analysing the level of coordination of two independent operators. Fitts' law is used to compare the quality of movement with or without CGC. A total of 40 subjects have been recruited for this study and the results show that the proposed CGC framework exhibits significant improvement (p<0.05) on all the motion indices used for quality assessment. This study demonstrates that visual guidance is an implicit yet effective way of communication during collaborative tasks for robotic surgery. Detailed experimental validation results demonstrate the potential clinical value of the proposed CGC framework. © 2012 Biomedical Engineering Society.link_to_subscribed_fulltex

The impact of expert visual guidance on trainee visual search strategy, visual attention and motor skills

Minimally invasive and robotic surgery changes the capacity for surgical mentors to guide their trainees with the control customary to open surgery. This neuroergonomic study aims to assess a &quot;Collaborative Gaze Channel&quot; (CGC); which detects trainer gazebehavior and displays the point of regard to the trainee. A randomized crossover study was conducted in which twenty subjects performed a simulated robotic surgical task necessitating collaboration either with verbal (control condition) or visual guidance with CGC (study condition). Trainee occipito-parietal (O-P) cortical function was assessed with optical topography (OT) and gaze-behavior was evaluated using video-oculography. Performance during gaze-assistance was significantly superior [biopsy number: (mean ± SD): control = 5.6 ± 1.8 vs. CGC = 6.6 ± 2.0; p &lt; 0.05] and was associated with significantly lower O-P cortical activity [ HbO 2 mMol × cm [median (IQR)] control = 2.5 (12.0) vs. CGC 0.63 (11.2), p &lt; 0.001]. A random effect model (REM) confirmed the association between guidance mode and O-P excitation. Network cost and global efficiency were not significantly influenced by guidance mode. A gaze channel enhances performance, modulates visual search, and alleviates the burden in brain centers subserving visual attention and does not induce changes in the trainee&apos;s O-P functional network observable with the current OT technique. The results imply that through visual guidance, attentional resources may be liberated, potentially improving the capability of trainees to attend to other safety critical events during the procedure

Optical imaging and spectroscopy for the study of the human brain: status report.

This report is the second part of a comprehensive two-part series aimed at reviewing an extensive and diverse toolkit of novel methods to explore brain health and function. While the first report focused on neurophotonic tools mostly applicable to animal studies, here, we highlight optical spectroscopy and imaging methods relevant to noninvasive human brain studies. We outline current state-of-the-art technologies and software advances, explore the most recent impact of these technologies on neuroscience and clinical applications, identify the areas where innovation is needed, and provide an outlook for the future directions